Adaptive and multiple time-scale eligibility traces for online deep reinforcement learning

Deep reinforcement learning (DRL) is one promising approach to teaching robots perform complex tasks. Because methods that directly reuse the stored experience data cannot follow change of environment in robotic problems with a time-varying environment, online DRL required. The eligibility traces method well known as an technique for improving sample efficiency traditional linear regressors rather than DRL. dependency between parameters deep neural networks would destroy traces, which why they are not integrated Although replacing gradient most influential accumulating gradients can alleviate this problem, operation reduces number reuses previous experiences. To address these issues, study proposes new be used even while maintaining high efficiency. When accumulated differ from those computed using latest parameters, proposed takes into account divergence past and adaptively decay traces. Bregman divergences outputs by exploited due infeasible computational cost parameters. In addition, generalized multiple time-scale designed first time. This design allows replacement (decayed)